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ABSTRACT 

In July 1994, College of the Canyons (COC) in 
California conducted several predictive validity studies of the 
College Board Assessment and Placement Services (APS) Reading Test. 
COC began using the 35-question objective format test in spring 1993. 
Fall 1993 test scores were used in determining the ability of the APS 
Reading Test to predict student success in both basic skills and 
college-level English courses offered at COC. Correlation 
coefficients were computed for each English course between the test 
scores and course grades with withdrawals converted to fails; and 
between test scores and course grades with withdrawals deleted. The 
median predictive validity value across all levels was .27, 
indicating a moderate linear relationship between test scores and 
course grades. This predictive validity coefficient value is below 
the .35 level recommended by the Chancellor's Office. In light of the 
myriad factors that influence grades other than placement tests, 
however, the obtained value of the predictive validity of the APS 
reading test was considered adequate to justify use of the test for 
placing students into reading courses at COC. Contains two 
references. (KP) 



*********************************^^ 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. 
********************************************** 



S 

*° College of the Canyons 

Q Santa Clarita Community College District 



PREDICTIVE VALIDITY STUDY 
OF THE 
APS READING TEST 

July 1994 



0 ... °*'^VXS%%^' 
^iass? i?5SS&5 

0»igin»t»"« it .mprove 
ftprodoction quWrty 



— " 7717.1 »?ed »n this docw 

OEP' po»'tion or P©«'CV 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



Office of Institutional Development 



8: 2 

\E MC BEST COPY AVAIIABU 



PREDICTIVE VALIDITY STUDY OF COLLEGE BOARD ASSESSMENT 
AND PLACEMENT SERVICES FOR COMMUNITY COLLEGES 

READING TEST 

This report summarizes the predictive validity studies carried out in July 1994. The 
College Board APS Reading Test, a 35-question objective format test, was examined to determine 
its ability to predict student success in both basic skills and college level English courses offered 
at College of the Canyons. 

The college began using the College Board APS tests in Spring 1993 and the data for 
these computations represent information from first-time students for three semesters (Spring 
1993, Fall 1993 and Spring 1994). 

Matriculation regulations require that community colleges which adopt a standardized test 
for making placement recommendations must qualify its use locally. The test must be a valid 
predictor of future course success given the unique features of the college curriculum, its 
instructors, and student population. Establishing a clear relationship between the test score and 
subsequent student performance in a particular course for which placement recommendations were 
made is predictive validity. A correlation of no less than +35 is needed to conclude that the test 
instrument hcs sufficient predictive validity for advisement purposes. 

TVsr.riptivft Statistics. The data collected for this study are summarized in Table 1. The mean 
score and the coefficient of skewness provide a clue about the relative degree of difficulty of die 
test. Positively skewed score distributions indicate that sampled students tend to find the test 
difficult. Thus, the APS Reading Test was clearly more difficult for students in English 034 than 
was the test for students in English 101. 



Table 1. 

Descriptive Statistics for APS Reading Test, 
By Course Level 



lest 


Course 


N 


Mean 


Median 


Standard 
Deviation 


Skegness 


Reading 


Engl. 010 


15 


8.13 


8.00 


4.61 


1.04 




Engl. 034 


88 


13.70 


14.00 


3.12 


1.27 




Engl. 080 


70 


19.19 


19.00 


3.56 


-0.05 




Engl. 101 


40 


26.18 


27.00 


3.59 


-1.01 
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Predictive Validity The primacy index of the strength of the relationship between test scores 
(predictor) and course grades (criterion) is the predictive validity coefficient, or the correlation 
between predictor and criterion* 

Two correlation coefficients indicating predictive validity were computed for each course 
- one between the test scores and course grades with all withdrawals (Ws) converted to Fail (F), 
and the second between the test scores and course grades with withdrawals deleted from the 
analysis. The resulting correlations are shown in Table 2 on the following page. Since the 
distribution of scores was restricted because the placement test scores were in use, the correlation 
values were adjusted by using a correction for restriction or range. Fall 1993 norms were used, 
with computations based on the entire population of tested students (N=l,644). The correction 
yields a better indication of the relationship between scores and grades by adjusting the value of 
the correlation coefficient to what it would be if the distribution of the predictor (APS test score) 
was unrestricted. 
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Table 2. 

Relationship Between APS Reading Test Scores 
and Grades, by Course 



r-Score to Grade 



Test 


Course 


W=Failure 


Ws Excluded 


Reading 


Engl. 010 


.72 


(15) 


•64 ( 9) 




Engl. 034 


36 


(88) 


36 (68) 




Engl. 080 


.14 


(70) 


.18 (62) 




Engl. 101 


-.07 


(39) 


-.64 (31) 



RESULTS AND DISCUSSION 
APS Beading TMt The median predictive validity value across all levels is .27, ranging from 
a low of -.64 to a high of .72. The correlation indicates a moderate linear relationship between 
the placement test scores and course grades. 

The predictive validity coefficient value of 21 is below the 35 level recommended by 
the Chancellor's Office, but is within the range reasonably expected for placement tests. There 
is a broad range of factors that are not, and can not be, tapped by the test, but which are related 
to performance in class. Among these are an assortment of individual difference variables, 
motivation level, and differences in background and experiences. Because there are such a 
complex array of factors that interact to determine performance in classes, many of which are 
non-academic, it is not reasonable to expect placement tests to be very strongly related to course 
grades. Indeed, the test's technical manual lists a median predictive validity of .28, with a range 
from -.01 to a high of 5\ (based on 64 validity studies, each of which had 100 students). 

The reader is reminded that these correlations apply only to the ability of the APS tests 
to predict the final course grade performance of first-time college students. 

CONCLUSIONS 

APS Rf aHit lg lea In light of the myriad of factors that impact course grades other than the 
specific skills measured by the test, and the unreliability of the criterion (course grades), the 
obtained value of the predictive validity of the APS Reading Test is adequate to justify use of 
the test for placing students into reading courses at College of the Canyons. 

The college should continue to evaluate the APS Reading Test and multiple measures 
which may provide the most accurate advice possible when directing students into classes. 
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